Server-based computing allows a networked client device, remotely situated with respect to a server computing system, to access computing resources on the server. For example, the client device may run desktop remoting client software and hardware that uses a remote desktop protocol, such as Remote Desktop Protocol (RDP), Virtual Network Computing (VNC), or Personal Computer over Internet Protocol (PCoIP), to access a desktop remotely. The desktop remoting client software displays an image of a graphical user interface generated by the operating system and applications running at the server computing system, e.g., in a virtual machine. The term “desktop” can refer to a virtual machine or physical system accessed by an end user as a local (to the user) desktop or workstation computer. The term “desktop” may also refer to the human interface environment through which users can launch, interact with, and manage applications, settings, and data. The remote desktop is accessible by the user using the remote desktop protocol and a desktop remoting client. The client device typically has a local set of input and output devices (e.g., display, keyboard, mouse) and transmits user input such as keyboard or mouse input to the remote system to be processed there and receives display and other data sound) for presentation to the user using the local set of I/O devices.
However, users of client devices with touch-screens face several challenges when interfacing with traditional desktop-based applications (e.g., Microsoft Windows® applications) and desktops through a remote display protocol. One challenge relates to latency of the network connection between the touch-screen device and the remote desktop, which often creates a halting or jittery user experience. Another challenge relates to attempts to remotely control a, conventional “point-and-click” driven interface (e.g., Windows® interface) from a touch-screen device which is designed not for traditional “point-and-click” interactions, but rather, for touch-screen gestures and “finger swipe” style of interactions.
To address these challenges, it is known in the art to modify a remote desktop controlled by touch screen devices, for example, by automatically adjusting a font size of the remote desktop. However, these largely cosmetic changes only modify the displayed contents received from the remote server, do not fail to reduce latency in menu navigation and user feedback, and still require users to navigate desktop applications through a “point-and-click” style interface.
It is also known in the art, as described in “Remote Desktop Protocol: Graphics Device Interface Acceleration Extensions” made available by Microsoft Inc., for a remote server to use its own video driver to render display output and encoding drawing operations that produce an image instead of encoding the actual image. For example, instead of sending a bitmap image of a filled rectangle from server to client, an order to render a rectangle at coordinate (X,Y) with given width, height, and fill color may be sent to the client. The client then executes the drawing order to produce the intended graphics result. However, this approach only produces a same corresponding graphical interface on the client as on the server, which are difficult to use on a client device having a touch-screen or non-pointer based style of interface.
Alternatively, it is also known in the art, as described in United States Patent Publication No. 2011/0314093 A1, to translate user gestures on a client device into a digital representation of a remote computer's native input format, such as a control common to a two-button mouse (e.g., click, window scroll, right click), thereby allowing the client device to control the computer. However, this approach forces a user to learn a new user interface and memorize which touch screen gestures is equivalent to a mouse input, which can be cumbersome hard to use. Further, this approach still requires capturing and displaying basic screen data from the remote desktop and does not improve latency in interactions between the user and the user interface.
Consequently, there is a need for improved systems and methods for providing access to a remote desktop having one style of user interface to a client device having a different style of user interface.